1. Field of the Invention
The present invention relates to a method, apparatus, recording medium, and computer program for processing signal and, in particular, to a method, apparatus, recording medium and computer program for smoothing an image other than a sharp edge thereof with the sharp edge maintained without destroying continuity and correlation of pixels.
2. Description of the Related Art
A contrast enhancement method using gradation conversion and a high frequency component enhancement method enhancing contrast in a high frequency region have been contemplated in a video camera as a method for improving contrast (lightness difference between darkness and lightness) and sharpness of an image (clearness of a boundary) of an image picked up by an image pickup device such as charge-coupled device (CCD), a complementary metal-oxide semiconductor (CMOS) device.
The contrast enhancement methods include a tone curve adjustment method converting the pixel level of each pixel of an image in accordance with a function having a predetermined input and output relationship (hereinafter referred to as a level conversion function), and a histogram equalization method adaptively varying the level conversion function in response to a frequency distribution of the pixel level.
The high frequency component enhancement methods include an unsharp mask method performing outline enhancement by extracting an edge from an image and enhancing the extracted edge.
In accordance with the contrast enhancement method, only a portion of the entire luminance dynamic range (difference between a maximum level and a minimum level) of an image can be enhanced. Furthermore, contrast is lowered in the lightest portion and the darkest portion of the image in the tone curve adjustment method, and in a luminance region having a low frequency distribution in the histogram equalization method. In the high frequency component enhancement method, only the contrast of a high frequency component of the image is enhanced, leading to an unnaturally enhancement in the vicinity of an edge. As a result, image quality is degraded.
Japanese Unexamined Patent Application Publication No. 2001-298621 discloses a technique that enhances a portion of an image other than an edge by amplifying the portion of the image other than the edge with the edge providing a sharp change in pixel value of input video data stored.
Each of known image processing apparatuses extracts only a structural component of a signal from input video signal (pixel value) using an ε filter, generates an amplitude component as a difference between structural components of the input signal, amplifies only the amplitude component, sums the structural component and the amplified amplitude component, and outputs the sum. When a video signal having slightly varying video signal components surrounding a sharp edge having a larger magnitude than a predetermined threshold value ε1 is input, the video signal is converted into the one with only that edge extracted.
The ε filter determines each pixel of the input image as a pixel of interest C, sets a tap formed of a plurality of adjacent pixels consecutively arranged in a horizontal direction centered on the pixel of interest (for example, two pixels L2 and L1 to the left of the pixel of interest, and two pixels R1 and R2 to the right of the pixel of interest), weight averages pixel values of the pixel of interest C and the plurality of adjacent pixels using tap coefficients (for example, 1, 2, 3, 2, and 1) in accordance with equation (1), and outputs conversion result C′:C′=(1xL2+2xL1+3xC+2xR1+1xR2)/9  (1)
If the absolute value of difference between the pixel value of the adjacent pixel and the pixel value of the pixel of interest is greater than a predetermined ε1 (for example, the adjacent pixel R2>ε1), the pixel value of the adjacent pixel R2 is replaced with the pixel value of the pixel of interest C as represented by equation (2):C′=(1xL2+2xL1+3xC+2xR1+1xC)/9  (2)
The video signal smoothed by the ε filter is also referred to as a structural component of the video signal.
The video signal processed by the ε filter is subtracted from the input video signal (identical to the video signal input to the ε filter), and a slightly varying video signal other than the structural component of the video signal is extracted and amplified. The slightly varying video signal other than the structural component is referred to as an amplitude component. A portion of the video signal amplified and output, other than the structural component, and the structural component of the video signal output from the ε filter are summed, and the resulting sum becomes a video signal with the sharp edge thereof maintained and the portion of the video signal other than the edge amplified.
If an edge having a sharp variation in pixel value is smaller in size than a predetermined threshold value ε1, the video signal output from the ε filter is a smoothed one with the sharp edge missing. Severe image quality degradation is caused in a pattern image if the pattern image contains infinitesimal edges.
Each of consecutively arranged signals is successively designated as a signal of interest. A plurality of adjacent signals are determined from signals consecutively arranged in one of a horizontal direction and a vertical direction with respect to the signal of interest. A difference between the signal of interest and each of the adjacent signals is calculated. The adjacent signals having a difference smaller than a first threshold value and the signal of interest are weight averaged to calculate an smoothed signal. The difference is compared with a second threshold value smaller than the first threshold value. Whether an infinitesimal edge is present in the vicinity of the signal of interest is determined depending on whether the difference is greater than the second threshold value. One of the smoothed signal and the signal of interest is selected depending on the determination result.
Since horizontal processing and vertical processing are independently performed in accordance with the above technique, appropriate extraction of the structural component becomes difficult.
In another technique, information of direction component (reference direction) different from the processing direction is used. Depending on determination as to whether the input signal is flat in the processing direction, whether to perform smoothing process is determined. A signal serving as an intermediate value between a value provided as a result of smoothing process and a value with no smoothing process performed is output depending on the degree of flatness.